the Claims: 

This listing of claims will replace all prior versions, and listings, of claims in the 
application: 

Listing of Claims: 

1 (currently amended). A method for structuring video by 
probabilistic merging of video segments, said method comprising the steps of: 

a) obtaining a plurality of frames of unstructured video; 

b) generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive 
frames; 

c) extracting a feature set by processing pairs of said segments, 
said extracting generating an inter-segment color dissimilarity feature and an 
inter-segment temporal relationship feature of each said pair of segments , said 
inter-segment temporal relationship feature including metrics of temporal 
separation between the segments of the respective said pair and accumulated 
duration of the segments of the respective said pair : and 

d) merging video segments with a merging criterion that applies a 
probabilistic analysis to the features of the feature set, thereby generating a 
merging sequence representing the video structure. 

2 (cancelled). 

3 (currently amended). The method as claimed in claim 23 
wherein the difference signal is based on a mean dissimilarity determined over a 
plurality of frames centered on one of the consecutive frames and corresponding 
in numb e r of framoa to a fraction of tho fram e rate of vidoo capture . 

4 (previously presented). The method as claimed in claim 23 
further including the step of morphologically transforming the thresholded 
difference signal with a pair of structuring elements that eliminate the 
presence of multiple adjacent shot boundaries. 




5 (original). The method as claimed in claim 1 wherein the 
processing of pairs of segments for visual dissimilarity in step c) comprises 
the steps of computing a mean color histogram for each segment and 
computing a visual dissimilarity feature metric from the difference between 
mean color histograms for pairs of segments. 

6 (currently amended). The method as claimed in claim 4- 7 
wherein the processing of pairs of segments for their temporal relationship in 
step c) comprises the processing of pairs of segments for a temporal 
separation between pairs of segments and for an accumulated temporal 
duration between of pairs of segments. 

7 (currently amended). The method as claimed in claim 1 A 
method for structuring video by probabilistic merging of video segments, said 
method comprising the steps of: 

a> obtaining a plurality of frames of unstructured video: 

b) generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive 
frames: 

c) extracting a feature set bv processing pairs of said segments, 
said extracting generating an inter-segment color dissimilarity feature and an 
inter-segment temporal relationship feature of each said pair of segments: and 

d) merging video segments with a merging criterion that applies a 
probabilistic analysis to the features of the feature set, thereby generating a 
merging sequence representing the video structure: 

wherein step d) comprises the steps of: 

generating parametric mixture models to represent 
class-conditional densities of inter-segment features of the feature set^ 
said parametric mixture models being statistical models : and 

applying the merging criterion to the parametric 
mixture models. 



8 (original). The method as claimed in claim 7 wherein step 
d) is performed in a hierarchical queue and comprises the steps of: 

initializing the queue by introducing each feature into the 
queue with a priority equal to the probability of merging each corresponding 
pair of segments; 

depleting the queue by merging the segments if the merging 
criterion is met; and 

updating the model of the merged segment and then updating 
the queue based upon the updated model. 

9 (original). The method as claimed in claim 1 wherein 
representing the merging sequence is represented in a hierarchical tree structure. 

10 (currently amended). A computer storage medium having 
instructions stored therein for causing a computer to perform the acts of: 

generating video segments from unstructured video by detecting 
shot boundaries based on color dissimilarity between consecutive frames; 

extracting a feature set by processing pairs of segments, said 
extracting generating an inter-segment color dissimilarity feature and an inter- 
segment temporal relationship feature of each said pair of segments; and 

merging video segments with a merging criterion that applies a 
probabilistic analysis to the features of the feature set, thereby generating a 
merging sequence representing the video structure; 

wherein said merging fiirther comprises the steps of: 

generating statistical models of the feature set: and 
applying the merging criterion to the statistical models . 

1 1 (currently amended). A method for structuring video by 
probabilistic merging of video segments, said method comprising the steps 
of: 

a) obtaining a plurality of frames of unstructured video; 

b) generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive video 
frames; 



c) extracting a feature set by processing pairs of segments, said 
extracting generating an inter-segment color dissimilarity feature and an inter- 
segment temporal relationship feature of each said pair of segments; 

d) generating a parametric mixture model of the inter-segment 
features comprising the feature se t, said parametric mixture model being a 
statistical model : and 

e) merging video segments with a merging criterion that applies 
probabilistic Bayesian analysis to the parametric mixture model, thereby 
generating a merging sequence representing the video structure. 

12 (original). The method as claimed in claim 1 1 wherein the 
processing of pairs of segments for visual dissimilarity in step c) comprises 
the steps of computing a mean color histogram for each segment and 
computing a visual dissimilarity feature metric from the difference between 
mean color histograms for pairs of segments. 

13 (currently amended). The method as claimed in claim 1 1 
wherein the processing of pairs of segments for their temporal relationship in 
step c) comprises the processing of pairs of segments for a temporal 
separation between pairs of segments and for an accumulated temporal 
duration betw ee n of pairs of segments. 

14 (original). The method as claimed in claim 1 1 wherein the 
parametric mixture model generated in step d) represents class-conditional 
densities of the inter-segment features comprising the feature set. 

1 5 (original). The method as claimed in claim 1 1 wherein step 
e) is performed in a hierarchical queue and comprises the steps of: 

initializing the queue by introducing each feature into the 
queue with a priority equal to the probability of merging each corresponding 
pair of segments; 

depleting the queue by merging the segments if the merging 
criterion is met; and 



updating the model of the merged segment and then updating 
the queue based upon the updated model. 

16 (original). The method as claimed in claim 1 1 wherein the 
merging sequence is represented in a hierarchical tree structure that includes a 
frame extracted from each segment and displayed at each node of the tree. 

1 7 (currently amended). A computer storage medium having 
instructions stored therein for causing a computer to perform acts for structuring 
video by probabilistic merging of video segments, the acts including: 

obtaining a plurality of frames of unstructured video; 

generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive video 
frames; 

extracting a feature set by processing pairs of segments, said 
extracting generating an inter-segment color dissimilarity feature and an inter- 
segment temporal relationship feature of each said pair of segments; 

generating a parametric mixture model of the inter-segment 
features comprising the feature set , said parametric mixture model being a 
statistical model : and 

merging video segments with a merging criterion that applies a 
probabilistic Bayesian analysis to the parametric mixture model, thereby 
generating a merging sequence representing the video structure. 

18 (currently amended), A method for structuring video by 
probabilistic merging of video segments, said method comprising the steps of: 

a) obtaining a plurality of frames of unstructured video; 

b) generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive video 
frames; 

c) extracting a feature set by processing pairs of segments, said 
extracting generating an inter-segment color dissimilarity feature and an inter- 
segment temporal relationship feature of each said pair of segments; 



d) merging adjacent video segments with a merging criterion that 
appUes a probabilistic Bayesian analysis to parametric mixture models derived 
from the feature set, said parametric mixture models being statistical models. 
thereby generating a merging sequence; and 

e) representing the merging sequence in a hierarchical tree 

structure. 

19 (original). The method as claimed in claim 18 wherein 
representing the merging sequence in a hierarchical tree structure includes 
displaying a frame extracted from each segment. 

20 (currently amended). A computer storage medium having 
instructions stored therein for causing a computer to perform probabilistic 
merging of video segments, said instructions performing the acts of: 

a) obtaining a plurality of frames of unstructured video; 

b) generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive video 
frames; 

c) extracting a feature set by processing pairs of segments, said 
extracting generating an inter-segment color dissimilarity feature and an inter- 
segment temporal relationship feature of each said pair of segments; 

d) merging adjacent video segments with a merging criterion that 
applies a probabilistic Bayesian analysis to parametric mixture models derived 
from the feature set, said parametric mixture models being a statistical models. 
thereby generating a merging sequence; and 

e) representing the merging sequence in a hierarchical tree 

structure. 

21 (currently amended). A method for structuring video by 
probabilistic merging of video segments, said method comprising: 

generating video segments from an unstructured plurality of video 
frames by detecting shot boundaries based on color dissimilarity between 
consecutive frames; 



extracting a feature set by processing pairs of segments, said 
extracting generating an inter-segment color dissirnilarity feature and an inter- 
segment temporal relationship feature of each said pair of oegmGnto;and segments, 
said inter-segment temporal relationship feature including metrics of temporal 
separation between the segments of the respective said pair and accumulated 
duration of the segments of the respective said pair ; 

merging the video segments with a merging criterion that applies a 
probabilistic analysis to the feature set, thereby generating a merging sequence 
representing the video structure, the merging being independent of any empirical 
parameter determination; and 

generating a hierarchy with the merged video segments, the 
hierarchy having a merging sequence represented by a binary partition tree. 

22 (previously presented). The method as claimed in claim 21 
wherein the merging the video segments includes maximizing the a posteriori 
probability mass function of a binary random variable that represents inter- 
segment features of the video segments. 

23 (previously presented). The method as claimed in claim 1 
wherein step b) comprises the steps of: 

generating color histograms from the consecutive frames; 

generating a difference signal from the color histograms that 
represents the color dissimilarity between consecutive frames; and 

thresholding the difference signal based on a mean dissimilarity 
determined over a plurality of frames, thereby producing a signal that indicates an 
existence of a shot boundary. 

24 (new). The method as claimed in claim 1 wherein said 
extracting of said inter-segment temporal relationship feature of each said pair of 
segments including determining a number of frames separating the respective said 
pair of segments and determining an accumulated number of frames in said 
segments of the respective said pair of segments. 



25 (new). The method as claimed in claim 1 wherein step d) 
comprises the steps of: 

generating parametric mixture models to represent class- 
conditional densities of inter-segment features of the feature set, said 
parametric mixture models being statistical models; and 

applying the merging criterion to the parametric mixture 

models. 

26 (new). The method as claimed in claim 25 wherein step d) 
is performed in a hierarchical queue and comprises the steps of: 

initializing the queue by introducing each feature into the 
queue with a priority equal to the probability of merging each corresponding 
pair of segments; 

depleting the queue by merging the segments if the merging 
criterion is met; and 

updating the model of the merged segment and then updating 
the queue based upon the updated model. 

27 (new). A method for structuring video by probabilistic merging 
of video segments, said method comprising the steps of: 

generating video segments from a plurality of frames of 
unstructured video by detecting shot boundaries based on color dissimilarity 
between consecutive frames; 

computing an inter-segment color dissimilarity feature and an 
inter-segment temporal relationship feature of each said pair of segments, said 
inter-segment temporal relationship feature including metrics of temporal 
separation between the segments of the respective said pair and accumulated 
duration of the segments of the respective said pair; and 

d) merging video segments with a merging criterion that applies a 
probabilistic analysis to said features, thereby generating a merging sequence 
representing the video structure. 



28 (new). The method of claim 27 wherein said computing of said 
inter-segment temporal relationship feature of each said pair of segments further 
comprises determining a number of frames separating the respective said pair of 
segments and determining an accumulated number of frames in said segments of 
the respective said pair of segments. 

29 (new). A method for structuring video by probabilistic merging 
of video segments, said method comprising the steps of: 

obtaining a plurality of frames of unstructured video; 

generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive 
frames; 

extracting an inter-segment color dissimilarity feature and an inter- 
segment temporal relationship feature of each said pair of segments, said 
extracting of said inter-segment temporal relationship feature of each said pair of 
segments including determining a number of frames separating the respective said 
pair of segments and determining an accumulated number of frames in said 
segments of the respective said pair of segments; and 

merging video segments with a merging criterion that applies a 
probabilistic analysis to the features of the feature set, thereby generating a 
merging sequence representing the video structure. 
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